(1) Field of the Invention
The present invention relates to an image coding apparatus for coding data related to a moving image and an image decoding apparatus for decoding the coded data, and particularly to an image coding apparatus and an image decoding apparatus which corresponds to the H.264/AVC standard.
(2) Description of the Related Art
Along with the development of digital technology, technology for coding images has also progressed and developed. However, since the amount of data in video information, especially moving picture information, is extremely great, the data transfer amount becomes extremely large when a coded digital video is transferred by media such as broadcast or Digital Versatile Disk (DVD). In particular, the amount of data in recently implemented hi-resolution broadcasts is six times the amount of data in a conventional Standard Definition (SD) video.
Along with development in digital video technology, technology for compressing data in order to process increasing amount of data has been used and developed for digital video data, however this development is made possible by compression technology which utilizes video data properties. Along with the increase in information process ability for computers, complex calculations can be performed for compression technology and video data compression rates have significantly increased. For example, the compression technology utilized in digital high definition satellite broadcasting or terrestrial high definition broadcasting is a codec known as MPEG-2, which compresses digital high definition satellite broadcasts to approximately 1/30th of their size.
AVC/H.264 (H.264/AVC) which is standardized to be the next video compression technology in MPEG-2 is a standard which realizes a compression rate approximately twice that of MPEG-2. A multitude of compression technologies are included in AVC/H.264, and a high compression rate is realized by combining these technologies. As a result, the calculation amount significantly rises. Video data compressed according to MPEG-2 and AVC/H.264 is multiplexed according to a standard known as the MPEG-2 system, and transmitted. In the MPEG-2 system, two format types are defined: the transport stream (TS) and the program stream (PS).
Encoded image data and audio data is known as a bit stream, and a packetized bit stream is known as a Packetized Elementary Stream (PES). PES is multiplexed by either a transport stream or a program stream.
A transport stream can include more than one program inside the stream at the same time, and has a format designed for used in an environment in which transmission errors in digital broadcasting and so on may occur. In other words the transport stream is a stream in which the program information and PES are multiplexed and aligned into TS packets at fixed lengths of 188 bytes.
The program stream is a format designed for use in environments where errors will likely not occur such as recording media like a DVD, and is made up of one program. A header attached at the top of aligned plural PES is called a pack and plural aligned packs compose a program stream.
Here, a buffer model is defined in order to not overflow the decoder buffer when the image is decoded. Coded Picture Buffer (CPB) and Decoded Picture Buffer (DPB) are defined as buffer models in AVC/H.264. CPB is a buffer which holds a stream before it is inputted into the decoder, and the DPB is a buffer which holds a picture that has been decoded by the decoder. When decoding a picture, the data in the picture stored in the CPB is extracted and instantly decoded by the decoder and movement such as that stored in the DFB is anticipated.
There are two methods: Constant Bit Rate (CBR), in which the rate of the inputted data is fixed, and Variable Bit Rate (VBR) in which the rate of the inputted data varies. In either case, when the picture data inputted into the decoder is instantly extracted, the picture data is controlled during coding such that a CPB underflow is not generated. For CBR, the data inputted into the CPB is controlled so as not to overflow. For VBR, when the CPB is full, the data input can be temporarily paused and thus an overflow in the CPB is not of concern. The greatest value in the input rate for the CPB differs according to the coding conditions, however for example the value defined in the Read-Only Format of a Blue-Ray Disk is 40,000,000 bps for level 4.1.
On the other hand, entropy coding (variable-length coding) is a compression technology included in AVC/H.264. The two schemes CAVLC and CABAC are present as entropy coding schemes. When coding a DCT coefficient in Context-based Adaptive Variable-length Coding (CAVLC), coding is performed from the opposite direction of the scan direction using the column and level which is the length of repeating 0's in the variable-length coding table.
CABAC is an abbreviation for Context-based Adaptive Binary Arithmetic Coding, is a scheme for changing the appearance rate of a coding target which changes with time and is a widely known scheme for arithmetic coding. In CABAC, in addition to normal arithmetic coding, a context is attached to each section of code which must be compressed, and the appearance rate is modified for each context.
Coding in CABAC is divided mainly into two processes. The first is a process known as binarization which changes multi-value information which must be coded, known as Syntax, into binary data, the second is a process for performing arithmetic coding which calculates the context for the binarization data changed by binarization.
The arithmetic coding finds an appearance rate of 0/1 for each bit of binary data, performs a coding process and updates the appearance rate at the same time. As a result, the arithmetic coding is a process performed per bit and the calculation speed is normally 1 bit/1 clock.
Additionally, one picture which makes up the moving image is created from a value which shows the strength and coloration of the light which is known as a pixel, and composes a basis in which 16×16 pixels known as a macroblock are collected. Normal moving picture compression is performed on a one macroblock basis. In a high-definition image, one picture is formed of 1920×1088 pixels and 120×68 macroblocks.
Since most of the image compression processing is performed using repetition per macroblock, the image compression is performed by strongly considering the macroblock basis, and when the pipeline processing and so on is performed, the image compression processing generally functions as a pipeline per macroblock. In other words, the time for processing one macroblock is approximately fixed for any macroblock.
The code amount when one macroblock is compressed is determined according to the state of the pixel included in the macroblock, and normally the code amount of a macroblock with a uniform pixel value decreases, and when the pixel value notably varies in one macroblock, the code amount increases. This shows that in a screen, there are areas where coding is often generated and where coding is not generated.
The arithmetic coding process is the calculation speed for 1 bit/1 clock as described above. When the code amount of one macroblock is extremely large, sometimes the arithmetic coding process for one macroblock cannot be finished within the processing time for 1 macroblock. Further, when this kind of situation has continued, the data for which the arithmetic coding process cannot be performed is recorded as queued data. In order to avoid this situation, the arithmetic coding process is designed to function at high-speed.
However, since there are limits to the speed at which arithmetic coding is performed, there are methods proposed to solve this problem (see for example Patent Document 1, Japanese Laid-Open Patent Publication 2003-259370). In the image coding apparatus or the image coding method in Patent Document 1, the intermediate code (intermediate data) uniformly corresponding to the variable-length coding is temporarily stored in the buffer in the process for coding the inputted image signal, and the variable-length coding and processing other than the variable-length coding is performed without synchronization by performing variable-length code on the intermediate code.
However in the image coding apparatus realized based on the Patent Document 1, there is the problem that the capacity of the buffer in the entire image coding apparatus increases remarkably.
FIG. 1 is a block diagram which shows the structure of the image coding apparatus realized based on the above Patent Document 1.
The image coding apparatus includes an image coding processing unit 901, an intermediate buffer 902, a variable-length coding unit 903, a CPB 904 and a packet generating unit 905.
The image coding processing unit 901 generates intermediate data (intermediate code) by coding the image data, and stores the intermediate data in the intermediate buffer 902.
The variable-length coding unit 903 reads out the intermediate data stored in the intermediate buffer 902 and generates an image stream which is a bit stream by performing variable-length code on the intermediate data. Subsequently, the variable-length coding unit 903 stores the image stream in the CPB 904.
The packet generating unit 905 reads out the image stream stored in the CPB 904, generates and outputs PES by packetizing the image stream.
In other words, in the image coding apparatus based on the Patent Document 1, the buffer capacity increases remarkably since two buffers, the intermediate buffer 902 and the CPB 904, are necessary. As a result, there are problems such as the increased scale of the image coding apparatus, the increase in input/output bandwidth for the buffer and the increase in the processing amount for controlling the amount of data stored in the buffer.
Additionally, in order to correspond to the structure of the above image coding apparatus, the image decoding apparatus realized based on the Patent Document 1 includes a CPB for storing the image stream before decoding, a variable-length decoding unit which performs variable-length coding on the image stream and generates the intermediate data, an intermediate buffer for storing the intermediate data, and an image decoding processing unit which uses the intermediate data as input and performs decoding other than variable-length decoding.
Accordingly, in the image decoding apparatus realized based on the above Patent Document 1, there is the problem that the capacity of the buffer in the buffer increases remarkably in the same way as the image coding apparatus described above.
Thus, the present invention is realized in view of the above problems, with the object of providing an image coding apparatus and an image decoding apparatus in which the capacities of the buffers have been reduced.